Combining Local Appearance and Holistic View: Dual-Source Deep Neural Networks for Human Pose Estimation

We propose a new learning-based method for estimating 2D human pose from a single image, using Dual-Source Deep Convolutional Neural Networks (DS-CNN). Recently, many methods have been developed to estimate human pose by using pose priors that are estimated from physiologically inspired graphical models or learned from a holistic perspective. In this paper, we propose to integrate both the local (body) part appearance and the holistic view of each local part for more accurate human pose estimation. Specifically, the proposed DS-CNN takes a set of image patches (category-independent object proposals for training and multi-scale sliding windows for testing) as the input and then learns the appearance of each local part by considering their holistic views in the full body. Using DS-CNN, we achieve both joint detection, which determines whether an image patch contains a body joint, and joint localization, which finds the exact location of the joint in the image patch. Finally, we develop an algorithm to combine these joint detection/localization results from all the image patches for estimating the human pose. The experimental results show the effectiveness of the proposed method by comparing to the state-of-the-art human-pose estimation methods based on pose priors that are estimated from physiologically inspired graphical models or learned from a holistic perspective.Comment: CVPR 201

Co-interest Person Detection from Multiple Wearable Camera Videos

Wearable cameras, such as Google Glass and Go Pro, enable video data collection over larger areas and from different views. In this paper, we tackle a new problem of locating the co-interest person (CIP), i.e., the one who draws attention from most camera wearers, from temporally synchronized videos taken by multiple wearable cameras. Our basic idea is to exploit the motion patterns of people and use them to correlate the persons across different videos, instead of performing appearance-based matching as in traditional video co-segmentation/localization. This way, we can identify CIP even if a group of people with similar appearance are present in the view. More specifically, we detect a set of persons on each frame as the candidates of the CIP and then build a Conditional Random Field (CRF) model to select the one with consistent motion patterns in different videos and high spacial-temporal consistency in each video. We collect three sets of wearable-camera videos for testing the proposed algorithm. All the involved people have similar appearances in the collected videos and the experiments demonstrate the effectiveness of the proposed algorithm.Comment: ICCV 201

Visual Attention Consistency under Image Transforms for Multi-Label Image Classification

Human visual perception shows good consistency for many multi-label image classification tasks under certain spatial transforms, such as scaling, rotation, flipping and translation. This has motivated the data augmentation strategy widely used in CNN classifier training -- transformed images are included for training by assuming the same class labels as their original images. In this paper, we further propose the assumption of perceptual consistency of visual attention regions for classification under such transforms, i.e., the attention region for a classification follows the same transform if the input image is spatially transformed. While the attention regions of CNN classifiers can be derived as an attention heatmap in middle layers of the network, we find that their consistency under many transforms are not preserved. To address this problem, we propose a two-branch network with an original image and its transformed image as inputs and introduce a new attention consistency loss that measures the attention heatmap consistency between two branches. This new loss is then combined with multi-label image classification loss for network training. Experiments on three datasets verify the superiority of the proposed network by achieving new state-of-the-art classification performance

An asymptotically compatible probabilistic collocation method for randomly heterogeneous nonlocal problems

In this paper we present an asymptotically compatible meshfree method for solving nonlocal equations with random coefficients, describing diffusion in heterogeneous media. In particular, the random diffusivity coefficient is described by a finite-dimensional random variable or a truncated combination of random variables with the Karhunen-Lo\`{e}ve decomposition, then a probabilistic collocation method (PCM) with sparse grids is employed to sample the stochastic process. On each sample, the deterministic nonlocal diffusion problem is discretized with an optimization-based meshfree quadrature rule. We present rigorous analysis for the proposed scheme and demonstrate convergence for a number of benchmark problems, showing that it sustains the asymptotic compatibility spatially and achieves an algebraic or sub-exponential convergence rate in the random coefficients space as the number of collocation points grows. Finally, to validate the applicability of this approach we consider a randomly heterogeneous nonlocal problem with a given spatial correlation structure, demonstrating that the proposed PCM approach achieves substantial speed-up compared to conventional Monte Carlo simulations

Patterns of Immune Infiltration in Endometriosis and Their Relationship to r-AFS Stages

Background: Endometriosis (EMS) is an estrogen-dependent disease in which endometrial glands and stroma arise outside the uterus. Current studies have suggested that the number and function of immune cells are abnormal in the abdominal fluid and ectopic lesion tissues of patients with EMS. The developed CIBERSORT method allows immune cell profiling by the deconvolution of gene expression microarray data.Methods: By applying CIBERSORT, we assessed the relative proportions of immune cells in 68 normal endometrial tissues (NO), 112 eutopic endometrial tissues (EU) and 24 ectopic endometrial tissues (EC). The obtained immune cell profiles provided enumeration and activation status of 22 immune cell subtypes. We obtained associations between the immune cell environment and EMS r-AFS stages. Macrophages were evaluated by immunohistochemistry (IHC) in 60 patients with ovarian endometriomas.Results: Total natural killer (NK) cells were significantly decreased in EC, while plasma cells and resting CD4 memory T cells were increased in EC. Total macrophages in EC were significantly increased compared to those of EU and NO, and M2 macrophages were the primary macrophages in EC. Compared to those of EC from patients with r-AFS stage I ~ II, M2 macrophages in EC from patients with stage III ~ IV were significantly increased. IHC experiments showed that total macrophages were increased in EC, with M2 macrophages being the primary subtype.Conclusions: Our data demonstrate that deconvolution of gene expression data by CIBERSORT provides valuable information about immune cell composition in EMS

Baichuan 2: Open Large-scale Language Models

Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering. However, most powerful LLMs are closed-source or limited in their capability for languages other than English. In this technical report, we present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens. Baichuan 2 matches or outperforms other open-source models of similar size on public benchmarks like MMLU, CMMLU, GSM8K, and HumanEval. Furthermore, Baichuan 2 excels in vertical domains such as medicine and law. We will release all pre-training model checkpoints to benefit the research community in better understanding the training dynamics of Baichuan 2.Comment: Baichuan 2 technical report. Github: https://github.com/baichuan-inc/Baichuan

A trehalose biosynthetic enzyme doubles as an osmotic stress sensor to regulate bacterial morphogenesis

The dissacharide trehalose is an important intracellular osmoprotectant and the OtsA/B pathway is the principal pathway for trehalose biosynthesis in a wide range of bacterial species. Scaffolding proteins and other cytoskeletal elements play an essential role in morphogenetic processes in bacteria. Here we describe how OtsA, in addition to its role in trehalose biosynthesis, functions as an osmotic stress sensor to regulate cell morphology in Arthrobacter strain A3. In response to osmotic stress, this and other Arthrobacter species undergo a transition from bacillary to myceloid growth. An otsA null mutant exhibits constitutive myceloid growth. Osmotic stress leads to a depletion of trehalose-6-phosphate, the product of the OtsA enzyme, and experimental depletion of this metabolite also leads to constitutive myceloid growth independent of OtsA function. In vitro analyses indicate that OtsA can self-assemble into protein networks, promoted by trehalose-6-phosphate, a property that is not shared by the equivalent enzyme from E. coli, despite the latter's enzymatic activity when expressed in Arthrobacter. This, and the localization of the protein in non-stressed cells at the mid-cell and poles, indicates that OtsA from Arthrobacter likely functions as a cytoskeletal element regulating cell morphology. Recruiting a biosynthetic enzyme for this morphogenetic function represents an intriguing adaptation in bacteria that can survive in extreme environments